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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
BOARD OF PATENT APPEALS AND INTERFERENCES 

Qian, Richard et al. Group Art Unit: 26 1 1 

Serial No.: 09/455,964 Examiner: Huynh, SonP. 

Filed: December 6, 1999 

Title: METHOD OF CREATING A SEMANTIC VIDEO SUMMARY USING 

INFORMATION FROM SECONDARY SOURCES 

APPELLANT'S AMENDED BRIEF 

Chemoff Vilhauer McClung 8c Stenzel, LLP 

1600 ODS Tower 

601 SW Second Avenue 

Portland, Oregon 97204-3157 

December 1, 2006 

Mail Stop APPEAL BRIEF-PATENTS 

Commissioner for Patents 

P.O. Box 1450 

Alexandria, VA 22313-1450 

Dear Sir: 

BACKGROUND 

This brief is in furtherance of the Notice of Appeal, filed in this case on June 19, 

2006. 

The fees required under 37 C.F.R Section 41.20(b)(2), and any required petition 
for extension of time for filing this brief and fees therefore, are dealt with in the accompanying 
TRANSMITTAL OF APPEAL BRIEF. 

This brief contains these items under the following headings, and in the order set 

forth below: 
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I. 


Real Party In Interest 


II. 


Related Appeals and Interferences 


III. 


Status of Claims 


IV. 


Status of Amendments 


V. 


Summary of Claimed Subject Matter 


VI. 


Grounds for Rejection to be Reviewed on Appeal 


VII. 


Argimient 


VII. 


Claims Appendix 



The final page of this brief bears the practitioner's signature. 

REAL PARTY IN INTEREST 

The real party in interest in this appeal is Sharp Laboratories of America, Inc., the 
assignee of the captioned application. 

RELATED APPEALS AND INTERFERENCES 

There are no other appeals or interferences that will directly affect, be directly 
affected by, or have a bearing on, the Board's decision in this appeal. 

STATUS OF CLAIMS 

A. TOTAL NUMBER OF CLAIMS IN APPLICATION 

There are 12 currently pending claims in the application. 
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STATUS OF ALL THE CLAIMS 



Claims canceled: 



None. 



Claims withdrawn: 



None 



Claims pending: 



1-12. 



Claims allowed: 



None 



Claims rejected: 



1-12. 



C. 



CLAIMS ON APPEAL 



Claims 1-12 are on appeal. 



STATUS OF AMENDMENTS 

No amendment was filed after final rejection. 

SUMMARY OF CLAIMED SUBJECT MATTER 

The claimed subject matter is generally directed to an improved automated method for 
creating a semantic summary of a video that will be meaningful to a viewer of the summary. The 
claimed method identifies a domain to which the video belongs, as disclosed in FIGS 1 and 2 and 
the written specification at page 4 line 29 to page 5 line 3. Exemplary domains could include 
high-level domains such as news, sports movies, etc. or could be more specific, such as football, 
soccer, etc. See Specification at p. 5 lines 1-3. The domain of the video is then used to locate, 
fi-om a source other than the video, information related to the content of specific portions of the 
video. See Specification at p. 5 lines 6-20. This external information is then used to identify the 
related portions of the video corresponding to the external information and create a summary of 
the video. See specification at p. 5 lines 20-28. 
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A first embodiment of the claimed subject matter, and specifically claimed in independent 
claim 1 , relates to a method that first identifies a domain of the video and uses the domain to 
identify information about the video at a source other than the video, both as described in the 
preceding paragraph. From the external information, a datum is extracted that relates to a 
semantic event depicted in the video. For example, the specification at p. 5 lines 16-22 describes 
an example of locating a textual description of a soccer game that is shown in the video to be 
summarized. From this textual description, scoring information, such as game time, the scoring 
team, the jersey number of the scoring player, etc. can be identified through optical recognition or 
any other suitable technique. The datum extracted from the information external to the video is 
then used to identify the particular portion of the video to which the datum relates, a summary of 
that particular portion then being created in response to the extraction of the external information. 
See Specification at p. 6 lines 17-27. 

In a second claimed embodiment, specifically claimed in independent claims 7 and 12, 
and supported by the same portions of the specification cited in the preceding paragraph, a video 
abstraction method locates an index of the video to be summarized, and identifies a domain of the 
video. The domain of the video together with the index, is used to identify specific portions of the 
video to include in the created abstraction. 

In a third claimed embodiment, specifically claimed in independent claim 1 1 and also 
described in the portions of the specification cited earlier, the method first identifies a domain of 
the video, from which a textual summary of the video can be located. From this textual summary, 
a datum related to a semantic event relevant to the video summary can be extracted and used to 
locate content in the video that corresponds to the extracted datum. The corresponding content 
can then be extracted from the video for inclusion in a semantic summary of the video. 
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GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

The grounds of rejection presented for review are: (1) whether claims 1-12 are 
unpatentable under 35 U.S.C. 102(e) as being anticipated by Boyer et al., U.S. Patent Application 
No. 2003/0066085 (hereinafter Boyer). 

ARGUMENT 

REJECTION UNDER 35 U.S.C. 102(e) IN VIEW OF BOYER 
GROUP I -Claims 1-6 

All of the claims of Group I claim a method for creating a semantic summary of a video 
that and include the specific steps of: 

(a) using an identified domain of the video to locate information related to the video at 
a source other than the video; 

(b) extracting fi-om that external source a datum related to a semantic event that 
describes a portion of the video; 

(c) identifying the portion of the video to which the external datum relates; and 

(d) "in response to" the extraction of the datum "creating" a summary of the identified 
portion of the video. 

The claimed limitation "in response to" requires that the summary of the identified portion 
of the video must be created after the datum related to the portion of the video is extracted firom 
the source external to the video. The term "create" is defined in Webster's Third New 
International Dictionary of the English Language, 2002, as "to bring into existence" or to "make . 
. . for the first time." No reasonable interpretation of the limitations recited above would read 
upon Boyer. 
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Beyer discloses a programming guide by which available video content can be browsed by 
time, category, etc. Upon selection of a user, a screen of the programming guide may display 
prefabricated summary information related to a specific video. For example, the progranraiing 
guide may display the duration of the video, rating information, a text summary, etc. In addition, 
the display may include links to external web pages that include actor stills, clips of the selected 
video, interview segments, etc. The defining characteristic of all of the summary content 
disclosed by Boyer, is that they are prefabricated so as to be accessible to a user of the 
programrning guide. That is to say, the actor stills, interviews, and excerpted video clips have all 
been "created" long before a user navigates the disclosed programming guide, using creation 
methods not disclosed by Boyer. Hence Boyer does not disclose the limitation recited in 
independent claim 1, of "creating a summary of said identified portion of said video in response 
to the extraction of said datum." 

The Examiner argues that Boyer discloses "identifying a domain of a video" in the 
passages describing a user of Boyer' s prograrmning guide navigating the "time" "channel" and 
"category" domains of the available content. See Boyer at FIG. 15 and Office Action at p. 5, 9. 
The Examiner contends that Boyer discloses that the domain is then used to access information 
about the video at a source other than the video in the passages describing the user navigating to a 
web page displaying summary information that includes the datum of video clips, video 
interviews, etc. See Office Action at p. 5-6. The applicant does not quarrel with either of these 
contentions, but does note that, since the Examiner is reading the claimed "datum" as Boyer' s 
video clips, etc., the Examiner's later contention that Boyer also discloses using this "datum" to 
"create" a summary in response to the extraction of the datum is irrational, when the Examiner 
contends that the summary created in response to extracting the datum is in fact nothing more 
than textual descriptive information of the video displayed on the very programming guide page 
from which the clips, interviews, etc are accessed. Stated in another way, the Examiner is 
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essentially arguing that Beyer discloses that summary information such as a title or still images 
from the video, are created after extracting datum comprising video clips, interviews, actor stills 
etc. linked through the very web page that already displays the summary information that is to be 
created. The circularity of this logic is apparent. Boyer does not disclose creating a summary in 
response to the extraction of video clips, etc. Accordingly, the Examiner's rejection of the claims 
of Group I was improper. 

GROUP II - Claims 7-10 and 12 

The claims of Group II each claim a method for creating a video abstraction of a video and 
include the specific steps of: 

(a) using an identified domain of the video along with a located index of the video to 
identify portions of the video for inclusion in the video, where the index is located at a source 
other than the video; 

(b) extracting the identified portions of the video from the video to form the video 
abstraction in response to locating the index; and 

(c) displaying the video abstraction to the user. 

The Examiner's rejection of the claims of this group is even more circular than the 
rejection of the claims of Group I because the Examiner argues that the identification of video 
clips, video interviews, etc. that are available through a link on a web page of a programming 
guide, is used to "extracf the video clips, interviews, etc. from the video itself Setting aside the 
issue of whether an actor or director interview related to a video is something that is "extracted" 
from the video, the contention that Boyer discloses that video clips or other content is "extracted" 
from a video, by first accessing those video clips or other content in a programming guide is 
absurd. If the video clips are already available to be displayed to a user, there is no need to 
"extract" them from the video. Moreover, the Examiner, when rejecting the claims of this group, 
again irrationally contends that the ultimate abstraction displayed to the user is the title, actor still 
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information etc. available on the very page from which the "datum" of the video clips and 
interviews are accessed, making it improper for the Examiner to contend that the title and actor 
still summary information is created by accessing the links to the video clips etc. on the same 
page that the summary information is already displayed. Finally, the Examiner cannot plausibly 
read the claimed abstraction as title information, when the claims require that the abstraction 
comprise extracted portions of a video, i.e. it must be made of clips or frames of the video being 
abstracted. Accordingly, the Examiner's rejection of the claims of Group II was improper. 
Group III -Claim 11 

The claim of group II differs from those of Group I in that the latter requires that portions 
of a video identified as being related to a datum extracted from a location other than the video, be 
themselves summarized (see independent claim 1 which requires that the identified portion of the 
video be summarized). In contrast, claim 1 1 merely requires that the portions of the video 
corresponding to the extracted data be located and extracted for inclusion in a semantic summary, 
and need not be individually summarized. Nonetheless, all of applicant's arguments v^th respect 
to the Exaniner's rejection of the claims of Group I also apply to the rejection of the claims of this 
Group. Accordingly, the Examiner's rejection of the claim 1 1 was also improper. 

CONCLUSION 

The Examinees final rejection of claims 1-12 under 35 U.S.C. Section 102(e) should be 
reversed, and the claims should be found patentable. 

Respectfully submitted. 




Kurt Rohlfs 
Reg. No. 54,405 
Attorneys for Applicant 
Telephone: (503)227-5631 
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1. A method of creating a semantic summary of a video comprising the steps of: 

(a) identifying a domain of said video; 

(b) using said domain to locate information related to said video at a source 
other than said video; 

(c) extracting a datum related to a semantic event from said information, said 
semantic event describing a portion of said video; 

(d) identifying said portion of said video related to said datum; and 

(e) creating a summary of said identified portion of said video in response to 
the extraction of said datum. 

2. The method of claim 1 wherein said information is a textual simunary of events. 

3. The method of claim 1 wherein said information is included in a worldwide web 

site. 

4. The method of claim 1 wherein said information is included in an electronic 
programming guide. 

5. The method of claim 1 wherein said domain is identified from an electronic 
programming guide. 

6. The method of claim 1 further comprising the step of displaying said summary to a 



user. 



7. A method of abstracting video comprising the steps of: 

(a) locating an index of said video from a source external to said video; 

(b) identifying a domain of said video for creating a video abstraction; 
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(c) using said domain together with said index to identify portions of said 
video for inclusion in said video abstraction; 

(d) extracting said identified portions of said video fi-om said video to form 
said video abstraction in response to the location of said index; and 

(e) displaying said video abstraction to a user. 

8. The method of claim 7 wherein said index is included in a worldv^de web site. 

9. The method of claim 7 wherein said index is included in an electronic 
programming guide. 

10. The method of claim 7 further comprising the step of identification of said domain 
by a user of said abstraction. 

11. A method of creating a semantic summary of a video comprising the steps of: 

(a) identifying a domain of said video; 

(b) using said domain to locate a textual summary of said video; 

(c) extracting a datum related to a semantic event relevant to said video 
summary from said textual summary; 

(d) locating content in said video corresponding to said datum; and 

(e) extracting said content related to said semantic event from said video in 
response to extraction of said datum from said video for inclusion in said semantic summsiry 
including at least one portion of said video. 

12. A method of abstracting video comprising the steps of: 

(a) locating an index of said video in at least one of a worldwide web site and a 
programimng guide; 
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(b) identification of a domain of said video for creating a video abstraction by 

a user; 

(c) using said domain together with said index to identify portions of said 
video for inclusion in said video abstraction; and 

(d) extracting said identified portions of said video from said video to form 
said video abstraction. 
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EVIDENCE APPENDIX: 



None. 
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RELATED PROCEEDINGS APPENDIX: 

None. 
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